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Office Action Summary 



Application No. ||jfp 
09/224,583 



1 Applicant(s) 



Examiner 

Michael N. Opsasnick 



BEUTNAGEL ETAL 



Art Unit 

2655 



- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )S Responsive to communication(s) filed on 15 October 2002 . 

2a)D This action is FINAL. 2b)K This action is non-final. 

3)D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 
closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4® Claim(s) 1-34 and 36-70 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 1-28.31-34.36-40.43-51 and 57-65 is/are rejected. 

7) E3 Claim(s) 29.30.41.42.52-56 and 66-70 is/are objected to. 

8) Q Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
1 1 )□ The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)QAII b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 1 9(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 
Attachment(s) 

1 ) ^ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). . 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-1 52) 

3) □ Information Disclosure Statement(s) (PTO-1 449) Paper No(s) . 6) □ Other: 
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PTO-326 (Rev. 04-01) 
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DETAILED ACTION 



Declaration filed under 37 CFR 1.131 



1 . Examiner notes the Declaration filed under 37 CFR 1.131. The arguments, presented in 
the declaration, were taken into consideration; and the Merrill reference has been determined to 
be predated by the Declaration filed under 37 CFR 1.131. 



2. Claims 29,30, 41,42,52-56, and 66-70 are objected to as being dependent upon a rejected 
base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 



3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 



Allowable Subject Matter 



Claim Rejections - 35 USC §103 
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4. Claims 1-28,3 1-34,36-40,43-5 1,57-65 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over A pplicant's Admitted Prior Art in view of Bothe (Audio to Audio- Video 
Speech Conversion with the Help of Phonetic Knowledge Integration (IEEE, Jan. 1997) , 
A pplicant's Admitted Prior Art (the prior art sections of applicant's spec, pp. 1-3, 
pertaining to the MPEG 4 standard) teaches a TTS stream input into a decoder, thru a 
synthesizer, and into a compositor (see Fig. 1 of applicant's specification). Furthermore, 
applicant's admitted prior art teaches FAPS and a FACE model into a FRM to be input to 
the compositor (Fig. 1). Applicant's admitted prior art also teaches the definition of the 
FAP's (pp 2 of applicant's specification), including 10 groups of parameters (ranging 
from different parts of the face, along with viseme information). The information with 
respect to the FAP's is used along with decoded audio information, to produce an 
animated video synthesized with the audio (applicant's admitted prior art, pp. 3, lines 22- 
26). 

{As a side note, throughout applicant's admitted prior art description on pps. 1-6 
of the specification, applicant stresses that the novelty of the instant invention is the use 
of phonemes, derived from the "tts stream decoder" output, generating FAPS to be input 
into the FRM, for eventual synthesis (applicant's specification, pp 2 lines 3-5, pp 4 lines 
10-11, and pp. 5 lines 3-5; and pp 6 lines 1-6).} 

To summarize, applicant's admitted prior art teaches a systems with a TTS stream 
into a decoder, synthesizer, and compositor, along with a face model and FAPS into the 
compositor such that the output of the compositor is a synthesized audio/visual, wherein 



Application/Control Number: 09/224,583 Page 4 

Art Unit: 2655 

the timing of the audio and visual information is derived from the streams (applicant's 
admitted prior art, pp 3 lines22-28). 

Applicant's admitted prior art does not explicitly teach the use of phonemes, 
derived from the decoded output of the TTS stream, to provide more FAP information 

into the FRM (into the COMPOSITOR) for synthesis. Bothe (Audio to ) , however, 

teaches the derivation of phonemes from a TTS stream (using a transform) to influence 
the animation of the phoneme (Fig. 1, pp 1632, second column; pp 1635, second column 
- page 1637, end of second column before 'Conclusion and Future Work). Therefore, it 
would have been obvious to one of ordinary skill in the art of speech animation to modify 
the teachings of Applicant's Admitted Prior Art, as explained above, with phoneme based 
animation information because it would advantageously produce an illusion of a mouth 
moving to generate the sounds heard by synchronizing audio playback of a phoneme with 
display of a mouth shape corresponding to the phoneme, for application such as video 
telephones and lipreading training ( Bothe (Audio to \ abstract). 

As per claims 1 and 2,12,14,16,31,33 the combination of Applicant's Admitted 

Prior Art in view of Bothe (Audio to ) teaches a decoder responsive to an input signal 

comprising text and FAP information (applicant's admitted prior art), generating addition 
FAP information from phoneme information to add to the FAP information (applicant's 

' Bothe (Audio to V admitted prior art, as demonstrated above), and a face rendering 

module responsive to the face model and decoder output (applicant's admitted prior art) 
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As per claims 3,17,23,36, the combination of Applicant's Admitted Prior Art in 

view of Bothe (Audio to ) teaches FAP based command signals (applicant's admitted 

prior art) 

As per claim 4,34,37 the combination of Applicant's Admitted Prior Art in view 

of Bothe (Audio to ) teaches prosody and timing information (Bothe (Audio to ) , 

pp 1635, bottom paragraph to pp 1636, bottom) 

As per claims 5,24,25, the combination of A pplicant's Admitted Prior Art in 

view of Bothe (Audio to ) teaches basic FAPS (applicant's admitted prior art - non- 

viseme based, and other grouping of FAPS). 

As per claims 6-1 1,13,15,18,19,32, the combination of A pplicant's Admitted 

Prior Art in view of Bothe (Audio to ) teaches sound, encoded voice, sound 

synthesizer, text, video synthesizer, and face model information (applicant's admitted 
prior art). 

As per claims 26-28, 38-40, the combination of A pplicant's Admitted Prior Art in 

view of Bothe (Audio to ) teaches an interpolated signal including phoneme, timing, 

and command information ( Bothe (Audio to ) data structure, Fig. 12, the neural 

networked derived information, as applied to applicant's admitted prior art structure) 

As per claims 16,20-22,43 and 57, the combination of A pplicant's admitted prior 

art in view of Bothe (Audio to ) teaches receiving an input that includes text 

specification, FAP information, output synthesized voice (applicant's admitted prior art), 
phoneme output with addition information (applicant's admitted prior art modified by 
Bothe (Audio to ) teachings, Bothe (Audio to X Fig. 1, as pointed to above), 
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generating a sequence of facial animation parameters from signals of said second output 

(applicant's ' Bothe (Audio to Y admitted prior art), and rendering images from the 

output signal and combining the synthesized voice and images (applicant's fig. 1, FRM, 
synthesizer, and compositor). 

As per claims 44 and 58, applicant's admitted prior art teaches receiving the 
input from a remote locations via a network 

As per claims 45 and 59, the combination of Applicant's Admitted Prior Art in 

view of Bothe (Audio to ) teaches bookmarks marking phoneme boundaries (Fig. 12). 

As per claims 46 and 60, the combination of Applicant's Admitted Prior Art in 

view of Bothe (Audio to ) teaches information about the FAP's (applicant's admitted prior art 

of groupings of FAP's). 

As per claims 47 and 61, the combination of Applicant's Admitted Prior Art in 

view of Bothe (Audio to ) teaches amplitude information (applicant's admitted prior art, pp2 

lines 10-13 of applicant's specification) 

As per claims 48 and 62, the combination of Applicant's Admitted Prior Art in 

view of Bothe (Audio to ) teaches duration ( Bothe (Audio to ) , pp 1636, first column, 

frame length) 

As per claims 49-5 1 and 63-65, the combination of Applicant's Admitted Prior 

Art in view of Bothe (Audio to ) teaches a specified transition path, chosen by the animation 

module based on phoneme information via the bookmark (pp 1636, the computer animation 
based on the phoneme input sequence and key vectors and the MLP). 
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Response to Arguments 



5. Applicant's arguments with respect to claims 1-34,36-70 have been considered but are 
moot in view of the new ground(s) of rejection. 



Commissioner of Patents and Trademarks 
Washington, D.C. 20231 
or faxed to: 
(703) 872 9314, 

(for informal or draft communications, please label "PROPOSED" or "DRAFT") 
Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal 
Drive, Arlington. VA., Sixth Floor (Receptionist). 



7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michael Opsasnick, telephone number (703)305-4089, who is 
available Tuesday-Thursday, 9AM-4PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ms. Doris To, can be reached at (703)305-4827. The facsimile phone number for 
this group is (703)872-9314. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the Group 2600 receptionist whose telephone number is (703) 305-4750, 
the 2600 Customer Service telephone number is (703) 306-0377. 



Conclusion 



6. 



Amy response to this action should be mailed to: 



mno 
7/6/2003 



